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(54) Digital camera with sound recording 

(57) A digital camera 1 for capturing sound passag- 
es as well as still images has the following elements. It 
has a camera apparatus 21 for capturing still images. It 
also has a sound recording apparatus 101, 102, 103, 
106 for capturing sound passages. A memory 107 is 
used for retaining still images as still image files and 
sound passages as sound passage files. The user in- 
terface of the camera is adapted for selectively linking 



one or more still image files with one or more sound pas- 
sage files to form a linked group, and for selectively un- 
linking one or more files from a linked group. This equiv- 
alent treatment of still images and sound passages al- 
lows effective use of the camera for recording ambient 
sound together with images, and for flexible construc- 
tion of sound/image composites representing a user's 
experience in a rich and effective manner. 
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Description 



. [0001] The present invention relates to a digital cam- 
era adapted to allow recording of sound as well as still 
images.. In- particular - the. invention relates to a digital 
camera arrangement designed to support the construc- 
tion ot ditlerent combinations of sound and images. 
[0002] It is known that the combination oi a still image 
and sound is a valuable one for the user,- not only where . *. . 
the sound is provided as annotation, but also where the to 
sound is ambienl at Ihe lime of recording the still image. 
Examples of systems adapted for recording both still im- 
ages and. sound together. are disclosed in OS Patents. ... 
Nos. 3439598, 4270653. .4270654, ' 4905029 and- 

•5276472. A system for display and playback of compos- '-'5 
ite sound and still image is set out in European Patent 

/ Application : Pubiicatiori ^No^082 i\- • 

of sound and still image is also obtainable from certain 
digital . yideo.cameras^wjiich^haya-.capacity to capture ■. 

: still -images - an example is the Hitachi MP -EG t A." ' V; 20 

,[0003] A camera operating system which supports 
augmentation of still images with sound is Digita, pro- 
vided by Flashpoint Technology, Inc. Digita provides a 
menu-driven interface., and also scripts which allow con- 
struction of composite items at the camera (such as a 2s 
page written in HTML with camera-captured image in- 
serts). 

[0004] In existing products, the sound capture is gen- 
erally optimised for annotation, rather than for recording 
of ambient sound. Accordingly, the microphone is direct- 30 
ed towards the user rather than to the imaged scene, 
the sound is recorded after the image is captured, and 
the combination of media is limited to one sound clip per 
image. Although capture of ambient sound is a known 
use of cameras with sound capture features, existing ar- 35 
rangements still provide a simple relationship of asso- 
ciation between a single still image and a single piece 
of sound. 

[0005] It is desirable to improve the capture of sound 
together with still images to allow a rich combination of *o 
the two which will provide particular benefits to the user. 

Summary of the Invention 

[0006] Accordingly, the invention provides a digital *s 
camera comprising: camera apparatus for capturing still 
images; sound recording apparatus for capturing sound 
passages; a memory for retaining still images as still im- 
age files and sound passages as sound passage files; 
and a user interface adapted for selectively linking one so 
or more still image files with one or more sound passage 
files to form a linked group, and for selectively unlinking 
one or more files from a linked group. 
[0007] Still image files and sound passage files may 
be files of different types, in which case selective linking 5 $ 
between files may be recorded in an index file containing 
all linkage information. Alternatively, linked groups may 
have a specific multimedia filetype supporting still image 



tiles, sound passage files; and links therebetween: in 
this case, still images' and sound passages may also 
themselves be provided in this filetype, as" it possesses 
the inherent capacity to be either a still image file, a 
sound passage file; 5 or. both: *.. - ■ ■ 

[0008] By' treating the images and. sound as" equiva- 

. lent, forms of data in this way. rather .than merely. con r 
sidering a sound passage as an optional addition to a 

. still-image,, richer /possibilities, for creation ahd.editing of , 
associated sound and image data become available. 
This equivalence makes selection of sound and image 
to be matched together, straightforward: Advantageous^ 
ly, it also allows for creation, of more complex objects 
with a linked group' containing any of one still image-file 
with one or more sound passage files one sound pas 
sage file with one or more stiir image files or even a 
plurality of both sound passage files and still image files 
This flexibility is particularly, important where, ambient 
sound is to be captured for example; a desirable ap 

• ' proach. may be' to record' th e sound, and during record- 
ing of th e.sound. to capture a r large number of still imag- 
es, deciding at a later point which images to keep. With 
a system in which sound is provided through annotation 
of a single image, this desirable mode of operating is 
simply not possible. 

[0009] Advantageously, the user interface is adapted 
such that a user can select a first and second view of 
the image files, wherein in the first view a plurality of 
sound passage files, still image files or linked groups 
are displayed in an index format for browsing, and 
wherein in the second view a single sound passage file, 
still image file or linked group is displayed for review and 
editing (including modification of any link). It is helpful if 
in this second view an additional sound passage file, stil I 
image file or linked group is also displayable, and the 
user interface is adapted to allow linkage between the 
displayed tiles to form a new linked group. Another de- 
sirable feature is for the sound passage files and still 
image files of a linked group to be viewable, selectively, 
either as part of the linked group or independently of the 
linked group. Another useful feature is for the user in- 
terface is adapted so that the sound passage files, the 
still image files, and the linked objects are all presenta- 
ble to the user as a separate sequence, and that the 
user may select between these separate sequences for 
browsing. This collection of features allows effective 
browsing through a collection of captured data, and ef- 
fective combination and review of the data collected. 
[0010] It is particularly useful, especially in the case 
of more complex linked groups, for the digital camera to 
be adapted such that a still image file may be associated 
with a particular point of time or period of time in the 
sound passage of a sound passage file. This is valuable 
if images and sound passages can be captured contem- 
poraneously. In connection with this, it is desirable for 
there to be a means to mark a captured still image or a 
captured sound passage with the time of recording. This 
time stamping can either be absolute, or in the case of 
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contemporaneous recording,. relative (that is, an image ^ 
is stamped with its time of. recording relative to the start* 
of the sound passage). Contemporaneous recordings, 
of sound and still images maybe provided automatically .... 
as linked groups This structure allows valuable forms 
of image association to be created, such as an assem 

.blagepf still images appearing in a determined succes- . 
sign and spacing against a common passage of ambient 
■sound- -J; ... r ' • r -' • 

[0011] Iri a development of this approach, embodi- 
ments of a camera may include a further data recording 
device for recording of another data type, and wherein ... 
the memorydevice is adapted to record the other data •• 
type as "a further data type fife, and wherein the user 
interface js adapted for selectively linking one or more ; 
further data type files into a k linked group, and for selec- 
tively un! in kin g'bne or more : further data type files from, 
a linked group These further data types may be graphic 
annotat ion : (f or , examp I e hajidwr iting input) or video-. 

^olips/;7;.: :i ^- ; ^r-— ' ' ^i,;" - - 

Description of Figures and Specific Embodiment 

[0012] Specific embodiments of the invention are de- 
scribed below, by way of example, with reference to the 
accompanying drawings, in which: 

Figure 1 shows a system for the capture and repro- 
duction of sound and still images according to an 
arrangement proposed by the present applicant; 

Figures 2a,' 2b and 2c show front (image and sound 
capture) side, rear (user interface side) and top 
(capture button) views of a digital camera in accord- 
ance with an embodiment of the invention; 



Figures 3a to 3c show the user interface display of 
Figure 2 in editing mode, and illustrate selection of 
different data elements; 

Figure 4 shows the user interface display of Figure 
2 in editing mode, and shows an audiophoto con- 
sisting of a sound passage with three attached im- 
ages; 

Figures 5a to 5c show the user interface display of 
Figure 2 at three different stages during the playing 
of the audiophoto of Figure 4; 

Figure 6 shows the user interface display of Figure 
2 in editing mode showing an index image of cap- 
tured data; and 

Figures 7a and 7b show the user interface display 
of Figure 2 in capture mode, and illustrate respec- 
tively a viewfinder display and an index display. 

[001 3] Basic components in a system for providing an 
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image and related sound are shown in Figure 1 /A cam-* 
era 1 comprises a conventional 'digital camera" met ha--' -\ 
nism for- capturing a still image-together with a digital'; ;.• 
sound recording . : device for. recording a passage s of^ 
sound Sound recording mean s are conventional a mi - ? 
crop hone 101, an automatic gam controller '102. an an 
^log/digital converter 103 and a codec 106 -.where pjay^ • • 
back is also available; as.in the embodiment to be de-.;;'- ; ; 
- scribed below, a loudspeaker. 105 and a.digital/ahalog .4. - - \< 
-converter ib4*are v a1so : lDe provided" A : memory ; 107 r fbr " v '■^' y "--^-^- < - • 
storage of audio and image information is also required. . - 
[0014] • It should be noted that although the. term °im 
age' is used here-it does hot refer merely to a pictorial.- \. v v . 
image: It-may also relate to, for example, text captured 
1 i.n an image^format. Likewise, when the Jerm "camera 0 ~) 
,^:ik use§^ other ; d^ 
understood to fall within its scope in the context. of the ^ - T 
present invention . . . > ^ 

[0015] Whereas, the process of image and sound cap- v 
v;<ture'may'be.;ert^^ 
alog image recording medium (such as film) and an an-vfi;. ; 
alog sound recording medium are provided at the cam- 
era; in the context of the present invention digital capture 
of both sound and image is used. Both image and sound 
are digitized on capture and stored in an appropriate 
storage medium (for example on a conventional flash 
memory or an Iomega "Clik" disk ("Clik" is a trade mark 
of Iomega Corporation)). In this event the digital data 
can be provided to a digital processing means (such as 
personal computer 2). This is advantageous, as it allows 
for easy editing of both image and sound data. The im- 
age data can then be provided from the digital process- 
ing means as a printed image, and the passage of sound 
attached thereto in a digital electronic storage medium, 
such as a flash memory. This is best achieved by means 
of a printer 3. In printer 3, the image is printed in con- 
ventional manner and a sound reproduction device with 
the passage of sound recorded thereon is attached to 
the image. Advantageously, recording and attachment 
of sound is done in the printer itself ; but this step may * 
be carried out separately (for example, at the sound re- 
cording device 4, as is described further below). Alter- 
natively, it is possible in some embodiments for the im- 
age data and optionally the sound data to be provided 
directly from the camera to the printer (for example by 
infra red data transfer using technologies currently 
known for infra red printing from personal computers - 
such as under the IrTranP standard): these arrange- 
ments may require the processor in camera t to be pro- 
vided with a printer driver for the appropriate printer. 
[0016] The result is a tangible representation of the 
image with an electronic storage medium having the 
passage of sound recorded thereon attached to it: this 
is here termed an "audiophoto". To play the passage of 
sound, sound reproduction device 4 is employed. This 
can be connected to the electronic storage medium to 
enable information stored in the electronic storage de- . 
vice to be transferred to the sound reproduction device, 
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the sound, reproduction .device 4 containing, means to. 
convert the information received from the electronic 
storage device into the passage of sound. However, 
sound reproduction device 4 is adapted to be detacha- 
ble or otherwise remote from the printed image when no 
connection between the; electronic storage device and 
the sound reproduction .device is reguired. " : 
[0017] . Further details of these system elements are 
provided in copending European Patent Application No - 
98305436.2, the contents of which are incorporated by' 
reference herein to the extent permissible by law. A dig- 
ital camera 1 which is an embodiment of the present in- 
vention will now be discussed below, with reference to . 
Figures 2a to 2c. 

[001 8] Figure 2a shows the front (image capture) side 
of a digital camera. 1 according to an embodiment of the 

^present! invWntiqK;ln; basic . 
conventional handheld camera. This camera is opti- 

. mised.:f or. capture of' ambient sound,- rather tban;annq-; 

'' tation of images,-, so in [addition to a conventional digital 
camera apparatus for capturing still images shown here, 
by lens 21 , the front side of the camera 1 also has the 
microphone 101 of a sound recording apparatus (of the 
kind discussed with reference to Figure 1) for capturing 
sound passages. In a digital camera optimised for an- 
notation; this microphone 101. would be placed on the 
rear (user interface) side of the digital camera 1 - alter- 
natively dual microphones, or means to reorient the mi- 
crophone for capture from different directions at differ- 
ent times, may be provided. The captured images and 
sound are 'eventually stored (in formats and associa- 
tions discussed below in the context of the user interface ' 
of the digital camera) on a memory for retaining still im- 
ages as still image files and sound passages as sound 
passage files. This memory comprises a removable 
flash memory card 107 held in a slot on the side of the 
camera, and released from the slot by a button release 
22. 

[001 9] Figure 2b shows the rear (user interface) side 
of the digital camera 1 . The display 23 is (preferably) an 
LCD or other flat screen display. The display can be 
used either for image and sound capture (capture mode) 
or for image and sound editing (editing mode) - both cap- 
ture mode and editing mode are discussed further be- 
low. The display 23 is shown in Figure 2b in editing 
mode, with an audiophoto image shown with three dis- 
play elements: image element 231 , sound element 232, 
and selection element 233, all of which will be ad- 
dressed below in the discussion of editing mode. Switch 
26 is used to allow the user to toggle between capture 
mode and editing mode. Also provided on the user in- 
terface side of the digital camera is loudspeaker 1 02, so 
that audio playback is directed towards the user. 
[0020] The user interface controls 24 are provided on 
the user interface side of the digital camera. The user 
interface controls 24 are used to browse, organise and 
edit the captured images and sound, and in particular to 
allow selective linking of one or more still image files 
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with one or more, sound. passage. files to form a linked ; 
qrouo, and for selectively unlinking one or more files 
from a linked group. The specific functions of different 
buttons wili be described in greater detail in the discus- 
■ 'sion.of the. editing .mode; but a brief -summary, is as fol- 
lows:' " . . ;•• V- 7 r - 

data type selection -button 241 allows selection of 
image sound or both as ."active", 
deletion button 242 allows deletion of the current 
active element; 

linking/unlinking, button- 243 allows selected ele- 
. . ,ments to;be;linked or. unlinked;..,. A • ... 

marking button 244 allows a sound clip to be 

marked (for reasons discussed further below); ' 

forward button 245 allows a user to step forward 
\,Y throughout; data wh ile\b rqw.sin.g, or to play a . sound./ 

passage 

/backward button 246: 'allows- a :usei% to, step back-;, 
ward throughout data whjle browsing; and ; . 
stop button 247 allows a user. to stop playing. of a, 
sound passage. 



[0021] Figure 2c shows a top view of the camera 1 . 

25 The top of the camera is where data capture controls 25 
are located. Data capture controls 25 are essentially 
conventional: there is an image capture button 251 eas- 
ily operable with the right hand to record an image, and 
a dual state sound capture start button 252 easily oper- 

30 able with the left hand to begin recording of sound on a 
first button press to end recording of sound on a second 
button press. 

[0022] Editing mode will now be discussed with refer- 
ence to Figures 3 to 6. 

35 [0023] Figure 3a shows an audiophoto in editing 
mode. The audiophoto consists of a still image 231 and 
a sound passage (provided as a graphical representa- 
tion 232 of the sound). The sound passage is marked 
with a tag 31 to indicate the association of an image with 

to it - the position of the tag 31 is indicative of the time along 
the sound stream represented by the graphical repre- 
sentation 232. The sound passage is also marked with 
its duration 32. Selection element 233 indicates in Fig- 
ure 3a that the whole audiophoto is selected. The se- 

45 lection element can be toggled between the states of 
audiophoto/sound element/image element by pressing 
the data type selection button 241 . 
[0024] When the audiophoto is selected as in Figure 
3a, the options provided by the different user interface 

50 buttons are as follows. Forward button 245 causes the 
display to advance to, and play, the next audiophoto. 
Backward button 246 causes the display to move to, and . 
play, the previous audiophoto. Stop button 247 causes 
the sound passage to stop playing. Linking/unlinking 

55 button 243 is for linking or unlinking the displayed ele- 
ments: for an audiophoto, the display at any one time 
will be showing an image element 231 and a sound el- 
ement 232 that are linked to each other. Pressing the 
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• linking/unlinking button 243 with an audiophoto sel ected . -marker (not shown in Figure.3a or Figure 3b,- but-visible: --^/ : ;^---V^.^ : k.; 
will generally be used to enable the linkage between the as a visual symbol in i sound" record "'232j1n'the/sbund^ ■' ''V"r.'"' 
• image element 231 and.the. sound element 232 to be- , record at a time : corresponding *t ' 

broken the result could.be separated image and sound record at which .the marker was pressed ,When ihe V ^ ' . V ^ 
' . - separated audiophoto and sound, separated audiopho- 5 sound record is not playing pressing of the marker but- ^ * - 
to and image, or even two separated audiophotos. (if ' ton 244 places the sound clip into a state in which the ; : v y 

...both the jmage and the. sound have additional attach- markers can be used for. editing: 'Initially, a. passage in, - -. ^ 

ments) - it can also be used to modify the link. Both de- . the sound record up to the. point of the' first marker is 

V l Jetion and modification of links are discussed below with selected Pre^ \ Xl. 

reference to Figure 3b The deletion button 242 in this tion to the first manner itself and further pressing^ 
mode causes deletion of the selected audiophoto (con- button 245 advances selection to the passage of sound 
"v.i" • ; firmation by pressing a different designated key would ; between the.ftrst and second markers Pressing.the but 

_ v be required in an effective practical interface), and mark- ton again advances to the second marker and succes- = , " 
; ' ■ - -V ' • " er button 244 is used to mark the sound passage 232, sive presses alternate between selection- of markers 

■".as is discussed further below with reference to Figure 15 and intervening passa'gesuntir the end of the sound clip * v ' r 

- - , 3b > - is reached, at which point the. process starts again from 

[0025] Jf instead.the.sound passage is selected, as in - , the beginning of the clip In this way, either a marker 
^ ; Fiigure 3b then the meaning of the buttons will be ti\\- itself, or the sound passage between two markers; can , > 
Vv r : . ; ferent in some cases ^iThe forward button 245 and the be selected By use of the backward button 246 it is pos, , .... , . 

backward button : 246 now cause display and play, of the^ ^ . ; •'. 

:% . next and previous sound passage respectively. No; . . sages. in the opposite direction In this way : a marker or . " - . 

intervening passage may be selected and either deleted 
with delete button 242 (and so edit the sound clip itself) 
or used to set up a link with an image (by using linking/ 
25 unlinking button 243), the link thus establishing either 
when the image will display within the resulting audio- 
photo (if the link is with a marker) or over what period of 
time within the resulting audiophoto that the image will 
display (if the link is with an interval). 
30 [0027] The alternative to selecting the audiophoto or 
selecting the sound passage is to select the image, and 
this is shown in Figure 3c. The forward button 245 and 
the backward button 246 now cause display of the next 
and previous images respectively. No change is made 
3S to the displayed sound passage. Consequently, selec- 
tion of images only can be used to allow cycling through 
all the images for attachment to the presently displayed 
sound passage. Deletion button 242 is used to delete 
the selected image. Linking/unlinking button 243 is now 
to used essentially lor linking the selected image with the 
currently displayed sound, using principles discussed 
above in relation to Figure 3b. 

[0028] Although not shown here, practical embodi- 
ments will require a confirmation step for significant 
45 events, such as deletion of an audiophoto, image or 
sound clip. Advantageously, an "undelete" function will 
also be provided to allow deletions to be recovered to 
whatever extent is practical. 

[0029] A highly advantageous feature of the embodi- 
50 ment of the present invention is that it supports audio- 
photos that are more than a simple association of a 
sound passage with an image: in particular, an audio- 
photo may comprise a spund passage with one or more 
images, each of which has a relationship in time with the 
55 sound passage, or alternatively a collection of more than 
one sound passage with one or more images. An exam- 
ple of an audiophoto with a single sound passage but a 
plurality of images is shown in Figures 4 and 5. Figure 



change is made to the displayed image. Consequently, 
selection of sound passages only can be used to allow 
cycling through sound clips to find a clip for attachment 
to the presently displayed image. Stop button 247 is 
used to stop playback of a sound clip. Deletion button 
242 is used to delete the selected sound passage. Link- 
ing/unlinking button 243 is now used essentially for cre- 
ating or removing links between the selected sound and 
the currently displayed image. Pressing the button 243 
causes a link to be established - or if a link already exists 
between the sound passage and the image, for that link 
to be removed. This link may represent a relationship in 
time between the sound passage and the image: in par- 
ticular, for which parts of the sound passage the image 
is to appear. There may be a simple one to one relation- 
ship with no element of time (in which case, the image 
will simply be associated with the beginning of the sound 
passage), but there may be a sequence of images ap- 
pearing successively at different times during the sound 
passage. To support this, the user interface provides 
means on creation of a link to position that link within 
the sound passage. When the link is created, it will ap- 
pear provisionally at the start of the sound passage (un- 
less capture was contemporaneous - see below), and 
can be moved forward or backward along the sound 
passage by use of markers, as described below. In a 
variant of this approach, if capture of images and sound 
are time stamped (either absolutely, or if contempora- 
neously captured images and sound are time stamped 
relative to each other), the initial position of a link be- 
tween contemporaneously recorded sound and images 
can be at the actual point of capture of the image within 
the sound record. 

[0026] Use of marking within the sound record can be 
used to improve construction of the relationship be- 
tween images and sound. While a sound passage is 
playing, pressing of the marker button 244 places a 
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4. shows the initial,visw,of such an audiophoto, with the. , 
audiophoto selected but the sound hot playing. There 
are three images 42, 43, 44 associated with the sound 
passage represented by trace 41 ■ each of which are rep- . 
•resented in the trace 41 by a discrete clip .icon 4S : : 46, : . 
47. .The'" clip icons are located in positions corresponding- 
to the point in the. sound passage . at which they are .in- . 
tended to appear. The change, in the display- as the 
sound,passage*pla.ys is shown in Figures 5a to 5c 
[0030]" Clip icon 45 is located at the start of the sound " 
passage, so image 42 is displayed from the beginning 
of the sound passage. A visual, representation for the~ 
amount of the sound passage so far played is, provided 
by a background bar 46 behind trace 41. In Figure 5a, 
it can be seen^ that the sound passage is part played, 
' but that the second clip icon. 46 has not yet been 
reached In consequence the first image 42, is. being dis- 
played When- the second clip icon 46 position is 
reached,, then the second image, 43 will be -displayed. 
Figure 5b "shows display of. the second image. 43 at a 
point during playback of the. sound passage; after the 
second clip icon 46 has been reached but before the 
third clip icon 47 has been reached. After the third clip 
icon 47 has been reached, the third image 44 is dis- 
played. In this case, each of the images is associated 
with a point in time (indicating the beginning of display 
of that image) rather than a period of time (as would be 
the case if the image was linked, through use of mark- 
ers, with a part of a sound clip). There is no basic differ- 
ence between the two situations - however, linking with 
passages of time can be used to provide dark intervals 
between pictures, or overlaps. 

[0031] It is desirable for there to be a view of the var- 
ious available audiophotos and elements which renders 
it easier to compare different elements, or to move be- 
tween elements to be edited. Accordingly the editing 
mode may also provide an index view for viewing mul- 
tiple elements, or collections of elements. Such an index 
view 61 is illustrated in Figure 6. This index view in- 
cludes ail elements that are not part of an existing audi- 
ophoto : together with existing audiophotos. A selection 
frame 62 is provided to allow selection for viewing and 
playing (as in Figure 3a to 3c) of a particular element or 
audiophoto - such selection can be achieved through 
use of forward button 245 and backward button 246. De- 
lete button 242 is used to delete the elements within se- 
lection frame 62. An appropriate button combination is 
used to switch between single audiophoto view and in- 
dex view. When in index view, transition to single audi- 
ophoto view is achieved by pressing data type selection 
button 241 while stop button 247 is being pressed. The 
reverse transition from single audiophoto view to index 
view can be achieved by the same button combination. 
When switching from index view to single audiophoto 
view, the display 23 will change to contain the element 
or elements within the selection frame 62 in single au- 
diophoto view. 

[0032] It is possible from the index view to determine 



. whether a particular entry is an image v a sound passage,/ 
a simple audiophoto or a more complex audiophoto (hot 
a simple 1:1 relation of image and sound). Images'64. 
and sound passages 65 are. readily identifiable as. they'., 

5 contain no.souhd trace and no image respectively., Sim- 
ple audiophotos 63 can be seen to contain one sound 
trace and one image. More complex arrangements, 
such as a single image with two sound clips 66 or a sin- 
, gle sound clip with multiple images 67. are again clearly 

to identifiable as instead of providing a representation in- 
dicative of the start (or any other point) of the sound pas- 
; sage, an icon is provided lor every element. For exam- 
ple the image 66 with two-sound-clips has a single image 
shown. but two sound traces shown:- however, the image 

|5 67 with a single sound clip but three images shows-all 
three images in the image view 67 rather than just a 
'single image associated with a particular point in time 
The image view. is thus .an effective way to. determine 
..VVquickiy.the composition (at.leasUhe imagexomposition):-. 
.20 6f existing audiophotos, - . ' '"i T^,-: '.:?'-- : - V;". 
. [0033] Diff erent forms oi. image view are possible. An 
alternative could be to show alt images, or all sound 
samples, or all audiophotos, and to toggle between 
these alternatives with the data type selection button 

2S 241 . Another alternative would be to provide separate 
index displays of sound passages and images. Images 
and sound samples could be displayed according to 
their time of capture (particularly appropriate if some 
form of absolute time stamping is available). 

30 [0034] By pressing button 26, the camera is switched 
between editing mode as discussed above, and capture 
mode. The initial image in capture mode will be as for a 
conventional digital camera: a simple image indicating 
what would be recorded if at that point image capture 

35 button 251 were to be pressed. If the image capture but- 
ton 251 is pressed at this point, then a simple image 
results {like images 64 in Figure 6). However, the posi- 
tion changes when recording of a sound passage is in- 
itiated by pressing sound capture button 252. This caus- 
es the display to appear as shown in Figure 7a, with a 
sound bar 7 1 showing the build up of the sound passage 
(the length of sound so far captured, together with an 
indication that recording is in progress, are also provid- 
ed) and a viewfinder image 72 below. If sound recording 

45 stop button 253 is pressed before image capture button 
251 is pressed, then a pure sound passage will be re- 
corded, like sound passage 65 in Figure 6. However, if 
image capture button 251 is pressed while sound re- 
cording is building up, then a contemporaneous sound 

50 and image capture occurs. Advantageously this links 
the captured image and the sound passage 71 directly, 
with the link registered for initial purposes at the point 
during the sound passage at which the image capture 
occurred (of course, the time relationship in the link can 

55 be changed at a later point through editing). 

[0035] If an audiophoto is built up in this way, it is de- 
sirable for an alternative view to be available to illustrate 
all the elements of the audiophoto. Accordingly, if the 
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data type- selection button 241 is pressed during cap-,, 
ture,' then the image display is toggled between the 
viewfinder image 72.shown in Figure 7 a to a composite-, 
of captured image thumbnalls 73 : each marked with, 
their time of Recording relative to the start of the sound 
passage The representation of the sound passage 71 

Js- marked, in ,this view.' with. indicators 74 showing the 
time of;recording for •the/different images. 
[0036] *£ ln addition to. pure capture and pure review., it 
is desirable and possible to support an intermediate ar- 
rangement in which new audio or image data can be 
inserted selectively into the existing set, rather than sim- 
ply added to the end- 1 - .'- - • "• • 

- [0037] Insertion of such an additional image or sound 
segment is here carried out from the review mode, with; 
the appropriate context (audiophoto, sound clip, or irri- 

. age) for the insertion being selected in the single audi- 

Iqphotoyte^ 

. [0038] .kWhenthe insertion. context- is. a sound clip,:the .. 
user presses image capture button 251 to attach a new 

: phptb;to^e\c|ip, oppresses sound capture button 252 . 
twice to capture a new sound passage which is append- 
ed tothe end of the existing sound clip. If the user wishes 
to insert a new sound segment at an appropriate point 
during the existing sound passage : this can be done by 
advancing to and marking with marker button 244 the 
desired insertion point, and in that state capturing the 
new sound clip. 

[0039] When the insertion context is a still image with- 
out sound, the user can simply press the image capture 
button 251 to add a new image after the existing image, 
or use the sound capture button 252 to record an audio 
segment which is then attached to the selected image. 
[0040] When the insertion context is an existing audi- 
ophotO: the user can press the image capture button 251 
to attach a new image to the sound clip, creating or ex- 
tending a series of photos for the same clip. The point 
in the sound passage at which the new image is intro- 
duced can be automatically assigned, or manually po- 
sitioned by use of the marking button 244 as previously 
described. Alternatively, the sound capture button 252 
can be used to record a further sound segment, either 
to append to the audioclip, or to position within the au- 
dioclip by use of the marking button 244 as previously 
described. 

[0041] After capture, the following information is 
stored in the memory 107: images (stored as discrete 
files), sound passages (stored as discrete files), and 
timestamping and (where relevant) linking information. 
There are two possible approaches for this, either of 
which can.be used in satisfactory embodiments. The 
first is for the image and sound to be stored in their own 
separate data types, with ali linking, timing and selection 
information stored in a separate index file. This solution 
is probably the simplest to implement, but is not conven- 
ient for selective transfer of information from memory 
1 07 to another device. The second approach is for there 
to be a multimedia fitetype for audiophotos, supporting 
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•image, sound, timing and linking information. Jf such a . 
filetype is used, then this filetype can be used for all in-- 
formation ..captured (including images, -alone;. !and 
sounds alone) so all files processed by .the camera can 
be of the same filetype Th is solut ion ai lows each resu It- 
ant audibphoto to be freestanding as a single file with- 
out reference to an index, rendering ^selective transfers 
from memory 10.7 straightforward: This information can 
then be transmitted in anyof the paths shown in Figure 
V to a pe rsonal cbmp ut e r 2 (an effective iroute for com- 
plex audiophotos - which could then be transmitted, for 

.' example; by electronic mail); directly to a printer 3 or to 
a reader appliance;^ For. simple ; images ; sound pas- 
sages, or audiophotoe sending directly to a printer or 

• reader appliance as .'essentia Ify as described in copend 
ing European Patent Application No 96305436 2 but 
differences are; required if there are mult-pie images in 
an ' audiophoto. For. example; a complex audiophoto 

,:sent .directly. to a: printer cou^ 

^uafolement shown in a composite .form as shown in Fig^. : 
ure;6:; The Reader appliance 4. cpuld also 
with a composite readerA/iewer appliance (not shown) 
capable of displaying images and playing back sound 
to play back an audiophoto as captured or devised at 
the camera 1 : the functions to be provided need only be 
those offered by the user interface of the camera 1 (pos- 
sibly retaining the browsing features but not the editing 
features), though with preferably sufficiently high quality 
display and sound playback to provide a satisfactory ex- 
perience to the user. 

[0042] In addition to still images and sound, it is pos- 
sible for other data types to be captured at a digital cam- 
era. A digital camera can typically be adapted to capture 
and store short bursts of video without difficulty: such 
video, bursts may be provided in the same manner as 
sound passages as an augmentation of a still image. 
Similarly, means to provide a graphic input at the user 
interface (for example : a pen-based digitiser interface 
and means for recognising handwriting or a custom al- 
phabet such as Graffiti) may be used to provide graphic 
input to provide a similar augmentation. Such augmen- 
tations may be treated in essentially the same way as 
sound passages, and may be used as further (or con- 
ceivably alternative) elements in a linked group. 
[0043] • As the skilled man will appreciate, many vari- 
ations upon the features indicated above can be made 
in devising alternative embodiments of the invention. 
For example, the structural arrangement of the camera 
can be changed considerably - display 23 may be used 
only as a user interface display, with a separate view- 
finder being placed elsewhere on the camera, or a dif- 
ferent arrangement and type of controls can be em- 
ployed for the user interface. 
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Claims 

1 . A digital camera comprising: 
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camera apparatus. (or, capturing still images; r 
sound recording apparatus for capturing sound 

...passages;-...,.-. \ -.- . '?£ 
a memory lor retaining still images as'stiH image 
files and'sound passages as sound passage 
: '."files^ahd^"'- iv " ;"/.'-;V ' v "" / T' ^-;' v " : ." 
a user interface adapted for selectively, linking- , 
one or more still image files with one. or more 

, sound passage : files to form a linked group, and. . 
for selectively J unl inking one or more files from 
a linked group. 

A digital camera as claimed. in. claim: .1 ; wherein' a k: 
linked group may consist drone still image. file with - 
one or more sound passage files \\ '~:\v/'.- 75 

A digital camera as-claimed;in - claim .1 ...wherein a ;...•;,,> , ; *; 
linked group may. consist of one sound passage file 
with one or more .still image files • \, . . . ,! 

A digital camera as claimed in ; any,of claims 1 ..to 3. 
wherein the user interface is adapted such that a 
user can select a first and second view of the image 
files, wherein in the first view a plurality of sound 
passage files, still image files or linked groups are 
displayed in an index format for review, and wherein 
in the second view an individual sound passage file, 
still image file or linked group is displayed for edit- 
ing. 

A digital camera as claimed in claim 4, wherein in 
the second view an additional sound passage file, 
still image file or linked group is also displayable, 
wherein the user interface is adapted to allow link- 
age between the displayed files to form a new linked 
group. 

A digital camera as claimed in claim 4 or claim 5, 
wherein in the second view a further sound passage 
or still image may be captured and linked automat- 
ically to the individual sound passage file, still image 
file, or linked group displayed. 

A digital camera as claimed in any preceding claim, 
wherein the digital camera is adapted such that a 
still image file may be associated with a particular 
point of time or period of time in the sound passage 
of a sound passage file. 

A digital camera as claimed in any preceding claim, 
wherein the digital camera is adapted such that the 
camera apparatus may capture one or more still im- 
ages at the same time that the sound recording ap- 
paratus is capturing a sound passage. 

A digital camera as claimed in claim 8, wherein any 
still images captured at the same time as a sound 
passage are provided initially as part of a linked 



. . group with the sound passage. 



10 A digital camera as claimed in any preceding claim, * 

''. v fii'rtpift r V»nmnriclnn a timA rorftrHinri m'Aflhs trt mark '-hr v : — 




'recording 



A digital camera as claimed in claim ! 0 where de- 
pendent on claim 8 wherem-the time recording 
means is adapted to mark the time of capture of a 
captured still image relative to a contemporaneous- 
ly recorded sound passages - i: - . r ^'-^y^ry r^: 

A digital camera as claimed in any preceding claimr 
wherein the user interlace is adapted fprselectwe^ 
'deletion by a user/of a stilLimage .\fileV\a>pUnd:^^ri. 
' sage.file or part of. a sound passage file, or a linked 
group . * 



20 13 A digital camera as claimed in any preceding claim, 
V wherein the user interface is adapted such, that,; 
sound passage files and still image files of a linked 
group are viewable, selectively, either as part of the 
linked group or independently of the linked group. 



14. A digital camera as claimed in claim 13, wherein the 
user interface is adapted so that the sound passage 
files, the still image files, and the linked objects are 
all presentable to the user as a separate sequence, 
and that the user may select between these sepa- 
rate sequences for browsing. 

A digital camera as claimed in any preceding claim, 
wherein the memory retains linked groups as sep- 
arate sound passage files, still image files, and one 
or more index files containing linking information. 

A digital camera as claimed in any preceding claim, 
wherein the memory retains linked groups as mul- 
timedia files, containing sound passage informa- 
tion, still image information, and linking information. 

17. A digital camera as claimed in claim 16, wherein the 
sound passage files and the still image files are both 
multimedia files. 
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1 8. A digital camera as claimed in any preceding claim, 
further comprising a further data recording device 
for recording of another data type, and wherein the 
memory device is adapted to record the other data 
type as a further data type file, and wherein the user 
interface is adapted for selectively linking one or 
more further data type files into a linked group, and 
for selectively unlinking one or more further data 
type files from a linked group. 

19. A digital camera as claimed in claim 18, wherein the 
further data type is handwriting data, and the further 
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• data, recording device is a device for recording 
handwriting, in a digital representation. 

20 A dig ital camera as claimed in claim 18, wherein the . 

further data type is a video clip, and the further data s 
- ' recording device is a device for recording video 
• Zy ■■ ' ; clips. . - • • " • '„.;• .. 

( . v - 1 - y 21 A digital camera as claimed in any preceding claim, 

^- wherein the sound recording apparatus and the io 
camera apparatus are arranged for data capture in . 

; :v r .;V-V*V ^substantially common. direct ion, -\- 



v- • 22: A digital camera as claimed in any preceding claim; " v - 

; • ' V: -- wherein- the memory is a'flash memory. device. / ; is- 

23. A digital camera as claimed in any preceding claim, . 
wherein the digital camera is adapted to be hand- 
held by a user for still image and sound passage 

*' , ' - k capture. : - \ ' , %>-V 20. 

24. A program for operating the user interlace of a dig- 
ital camera comprising camera apparatus for cap- 
turing still images and sound recording apparatus 

for capturing sound passages, wherein the program 25 
comprises means for linking one or more still imag- 
es with one or more sound passages to form a 
linked group, and means for unlinking one or more 
elements from a linked group. 
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